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Parent-Focused Interventions: A Meta-Analytic 
Consideration of Risk and Outcome Categories 

The mid-1960s saw a revival of a longstanding idea: A way to intervene with 
infants and toddlers whose environment puts them at risk of suboptimal development is 
to intervene with their parents. Since then, numerous narrative reviewers have 
attempted to assess parent-focused intervention efficacy. A recent work, informed by 
the authors' clinical experience, was Olds and Kitzman's (1993) review of home visiting 
programs. The authors pointed to parents of low birthweight babies and adolescent 
parents as two groups that seemed especially to benefit from intervention, possibly 
because feelings of vulnerability as parents made them receptive to an offer of help. 
Among programs for parents of low birthweight babies, positive outcomes were notable 
in the areas of child development and of home environments that were stimulating. 

Olds and Kitzman’s (1993) review was unusual in encompassing several risk 
factors. Typically, a narrative review comprises one risk factor, such as low 
socioeconomic status, parental youthfulness, or an elevated risk of abuse and neglect. 
Reviews frequently suffer from an "overlumping" of outcomes, in which measures of 
disparate domains are put into one "good outcome" category. Even though Olds and 
Kitzman introduced more complexity than is usual in a narrative review, their work still 
reflected the limitations faced by a narrative reviewer, who can manage a limited 
number of studies and who is unable to detect any but the most apparent relations 
between variables. 

In contrast, by using coding for extracting potentially important information from 
the studies being reviewed, a meta-analytic reviewer can manage and manipulate large 
quantities of data. This enables the reviewer to make full use of outcome information 
and to explore interactions as well as main effects. For the present study, meta- 
analysis was used to test some of the ideas posited by Olds and Kitzman (1993), along 
with exploring the potential of meta-analysis for assessing the efficacy of interventions 
designed for parents of young, environmentally at-risk children. 

Method 



Forming the Sample 

An exhaustive literature search was carried out via the following means: 
electronic data bases, literature reviews, reference lists of studies, Internet resources, 
on-site searches of two archives, and contacts with program evaluators. Once 
unearthed, evaluations of parent-focused interventions were included in the analysis if 
they met all of the following criteria: 

• The program was implemented between 1965 and 1999. 

• The program was focused on the parent, the parent-child dyad, or the family, 
with a prominent or sole goal being the improvement of parental behavior or 
skills as a means of fostering the child's well-being. 

• Outcomes measured were of parental knowledge, attitudes or behavior, of the 
home environment, or of some aspect of the child's development or well-being. 
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• Outcomes were measured using public or medical records, observations, 
interviews, or paper-and-pencil tests. 

• The research design included a treatment group and a comparison group. 

• The risk factor or factors stemmed at least partly from the child's environment 
(e.g., economic deprivation; adolescent parents), not solely from biological 
factors (e.g., a low birthweight). 

• The program was preventive, not tertiary, in nature. 

• The "target child" was three years of age or younger at the beginning of the 
intervention. 

• An identifiable "dose" of intervention was delivered (or intended to be). 

• Outcomes were measured and reported in such a way that an effect size could 
be calculated for at least one outcome. 

These qualifications were met by 232 prevention projects which yielded 282 program 

evaluations (because some projects encompassed more than one discrete program). 

Other characteristics of the sample are portrayed in Table 1 . 

Risk Factors and Outcomes 



Parental risk factors around which programs had been designed fell into five categories: 

• Low socioeconomic status 

• Adolescence 

• A low birthweight baby (in conjunction with another risk factor) 

• An elevated risk of abuse or neglect, identified through a screening procedure 

• An emotional problem, including depression, anxiety, and substance abuse 

Outcomes were divided into six categories: 

• Child development 

• Home environment 

• Parental knowledge and attitudes 

• Dyadic interaction 

• Caretaking 

• Abuse and neglect 

Data Analysis 

After effect sizes were calculated as Hedges' g: Mi - M 2 / s (where Mi is the 
treatment group mean, M 2 is the comparison group mean, and s is the pooled standard 
deviation), the correction factor J was applied to adjust for bias caused by small 
samples (see Hedges & Olkin, 1985). When an outcome domain had been assessed 
in a study with more than one measure, an aggregate effect size for that domain was 
calculated. Within each study, a composite outcome score was calculated by 
aggregating across all six outcome domains. 

Analyses were carried out with a goal of examining the relations between and 
among groups on the basis of mean effect sizes. Confidence intervals were used for 
assessing the significance of differences between groups, as recommended by Durlak 
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(1995). Because a confidence interval specifies all the values of the sample mean for 
which the null hypothesis ought to be retained, two means can be deemed significantly 
different from each other if their confidence intervals do not overlap. 

For this study, 90% confidence intervals were constructed. A 90% confidence 
interval entails a 10% (or .1) probability that the population mean falls outside the 
interval constructed around the sample mean. When assessing the difference between 
two such means by examining their confidence intervals for overlap, the probabilities 
are multiplied, meaning that the probability of erroneously rejecting the null hypothesis 
(that the means do not differ) is .1 multiplied by .1, or .01, yielding a conservative test of 
significance. 



Results 

Effect sizes by outcome domain for all risk categories combined are shown in 
Table 2 and in Figure 1 . Outcome domain effect sizes are then shown separately for 
the risk-factor categories of low socioeconomic status, parental adolescence, elevated 
risk of abuse and neglect, and low birthweight baby (see Tables 3 through 6 and 
Figures 2 through 5). 

Programs for parents with suboptimal emotional functioning yielded too few 
studies for a domain-by-domain evaluation of effect sizes. An assessment of the 
composite outcome variable revealed an effect size of .03, which was based on 14 
studies yielding a total n of 592. 



Discussion 

Answers to questions about possible impacts of parent-focused interventions 
may vary on the basis of risk factors and of outcome domain. In providing evidence of 
this possibility, the results of the present study illustrate a strength of the meta-analytic 
approach. It can pick out significant patterns, across many studies, that would be 
difficult if not impossible to see using the limited number of studies and the box-score 
approach of a narrative review. 

In fact, anything a narrative review can do, a meta-analytic review can do, and 
often better. Moreover, when used as a narrative-review follow-up, a meta-analysis can 
help resolve contradictions or can find empirical support for conclusions, based on a 
limited sample, that were somewhat tentative. Such was the case in the present study, 
in which the results fell into a pattern aligning with Olds and Kitzman's (1993) astutely 
insightful ideas about the risk groups and outcome domains in which intervention 
effects may be greatest. 

When pondering meta-analytic results, however, it is important to be aware of 
the technique's limitations. Some limitations stem from the fact that a meta-analysis is 
very much like a primary research study, the fundamental difference being that meta- 
analytic subjects are research studies rather than people. Consequently, a meta- 
analysis is plagued by most of the problems to which primary studies are prey. For 
example, if only a few of the subjects divulge information about an aspect of 
themselves, the researcher cannot include that aspect in the study's variables. One 
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such aspect of the programs I reviewed was staff retention over the life of the program, 
which was mentioned in only a smattering of program evaluation reports. 

In addition, a few unique problems inhere. A major one stems from a seemingly 
simple fact which has far-reaching ramifications for the meta-analyst. His or her 
"subjects" have decided, a priori, exactly which pieces of information about themselves 
are worth revealing. In other words, if a variable of interest to the meta-analyst was not 
measured in any of the studies being analyzed, or if the variable might have been 
measured routinely but was never reported, then the variable cannot be included in the 
analysis no matter how important it might possibly be. 

These limitations seemed especially acute in the present study which, like meta- 
analyses commonly conducted in the field of medicine, was an assessment of the 
apparent effects of a treatment. A difference is that in a medical endeavor, with a goal 
of affecting physical functioning, a treatment may exert its effects primarily through 
chemical or mechanical means. In the controlled setting of a medical experiment, it can 
be a straightforward matter to describe accurately the treatment that was administered. 

Matters are not so straightforward in interventions for “high-risk” parents, which 
usually have a goal of affecting cognitions and behavior. That means that the treatment 
is inextricably entwined with the personalities of those providing and receiving the 
treatment, and an interpersonal relationship may be an integral part of the treatment 
itself. Indeed, theory and research in more than one area of the social science 
literature suggests that the quality of parent-intervenor relationships may be of 
paramount importance in determining intervention effectiveness. 

It follows that for a meta-analysis of parent-focused interventions, every program 
evaluation ideally would include variables related to intervenors' and parents' baseline 
interpersonal and intrapersonal characteristics, intervenors' working conditions, and the 
nature of intervenor/parent relationships. In reality, as suggested earlier, even proxy 
variables in some of these areas were so seldom available to me as to be unusable. 

Such lacks made themselves felt as I formulated research questions, and they 
also made themselves felt as I pondered my meta-analytic results. As is true in any 
quantitative study, a statistically significant result brought on a familiar headache, in that 
the result was no more than a significant correlation that needed to be interpreted. The 
throbbing was made worse by the limited range of possible mediator or moderator 
variables at my disposal. Variables falling in the areas of structural characteristics of 
the programs and demographic characteristics of parents and intervenor, which were 
available in enough evaluations to be usable, were limited in their utility. 

The conclusions I could draw were limited further by the substantial amount of 
inaccuracy that I knew surely was in my data. Here, too, there are problems uniquely 
inherent to the meta-analytic approach. The data for an analysis of the sort I conducted 
are collected, coded, analyzed, and reported by as many research teams as there are 
research projects represented in the analysis (232 in mine), with each team having 
operated independently of the analyst and of each other. Atop all that potential for 
inaccuracy to creep in, the meta-analyst adds mistake-making potential. Program 
characteristics must be represented by numeric codes, which entails a surprising 
amount of estimating and educated guessing. As for calculating effect sizes: A lengthy 
paper could be written explaining why even the most fastidiously careful coder finds it 
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distressingly easy to make mistakes (and why it is not possible to goof-proof the 
process by mechanizing it). 

This is not to say that the results of my meta-analysis should be seen as 
unsound. I feel confident that any inaccuracies in the data are peppered evenly across 
the categories of my data base. As long as we stay within the data base, then 
pondering the relative relations of effect sizes across categories seems legitimate and 
possibly instructive in modest ways. 

I only hope that more will not be made of the results than is warranted. In the 
earliest stages of the study, I had assumed that at study's end, I would draw some sort 
of conclusion about program efficacy. I now am unwilling to do so, partly because of 
unanswered questions about measuring the effects of parent-focused interventions. 
Questions include ones about the validity of measures and about the point in an 
intervention or in a family's development at which intervention effects are likely to 
become manifest. 

If problems of inaccuracies in the data and questions about outcome 
measurement were not enough to stop me from addressing the question, "Do these 
programs 'work'?", then I would be stymied by my inability to define "these programs." 
My data base includes evaluations of skillfully carried out interventions that probably 
turned some families’ lives around, along with ineptly carried out interventions that 
might have done more harm than good, and I have no more than a few hints about 
which is which among the 282 interventions. Because I have virtually no information 
about parents and intervenor at baseline and across the weeks or months of their 
relationship, then asking me, "Do these programs 'work'?" would be akin to asking me 
to analyze the effects of a drug when I had only the haziest idea of what was in the 
capsules that the research participants had swallowed every day. 

If the present study illustrates any point with special clarity, it is that statistical 
power is not synonymous with explanatory power. True, a meta-analysis can bring an 
impressive amount of statistical power to bear on the research questions at hand, and 
yet it will be no better and no more instructive than is the information that went into it. 
Although it seems to be commonly assumed that a meta-analyis produces a 
comprehensive overview of an area of the research literature, its view, when compared 
with that of a primary research study, can be much more limited in important ways. 

What a meta-analysis can do, in areas in which there is adequate information for 
coding, is take a broad sweep through the data and suggest fruitful directions for more 
fine-grained studies. In this and in other ways, meta-analytic results can be useful. 
While pondering such results, though, it is wise to remember a comment that Green 
and Hall made (1984, p. 52) while considering the strengths and weaknesses of the 
meta-analytic approach: "Statistical methods, to be useful, must be used thoughtfully, 
[for] data analysis is an aid to thinking, not a substitute." 
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Table 1 



Characteristics of the Sample 



Characteristic 


Nature of the Evaluation Report 




Published 


159 


Dissertation - results not published 


35 


Unpublished in-house document 


70 


Unpublished - other types 


12 


Produced too recently to be categorized 


6 


Year Evaluation Outcomes First Were Reported 




1970-1974 


26 


1975-1979 


25 


1980-1984 


53 


1985-1989 


47 


1990-1994 


61 


1995-2000 


70 


Risk Factors for Which Programs Were Designed* 




Low SES 


124 


Adolescence 


89 


Criteria met for risk of abuse or neglect 


52 


Emotional problem 


14 


Low birthweight baby/second risk factor 


13 


Assignment to Treatment and Comparison Groups 




Random 


145 


Not random 


133 


Unknown 


4 


Site 




Participating parents' home 


131 


Site established specifically for the intervention 


59 


Hospital or clinic 


39 


Other "everyday" locale (e.g., church; school) 


26 



* Note. Program evaluations, categorized on the basis of risk factors, sum to more 
than 282 because some programs were designed for parents with two concurrent 
risk factors. 



(Table 1 continued ) 



Characteristic 




School or program for pregnant/parenting teens 


23 


Mobile unit (bookmobile-like in nature) 


1 


Unknown 


3 


Delivery Mode 


Individual 


161 


Group 


54 


Both 


64 


Unknown 


3 


Program Approach 


Primarily didactic 


88 


Primarily supportive 


17 


Both didactic and supportive 


175 


Unknown 


2 


Intervention Duration (in weeks) 


Less than 1 


16 


2-13 


51 


14-52 


59 


53-104 


61 


105-259 


46 


260 or more 


38 


Unknown 


11 


Intended Number of Intervention Contacts with 
Participating Parents 


1 


13 


2-8 


51 


9-24 


54 


25-50 


45 


51-100 


38 


101-200 


38 


More than 200 


22 


Unknown 


21 



8 
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Table 2 



Mean Effect Sizes and Confidence Intervals bv Outcome Domain 



Outcome Domain 


Mean ES 


90% Cl No. of Studies Total N 


Child Development 


.28 


.25 - .31 


162 


16,615 


Home Environment 


.22 b 


.20 -.24 


122 


21,941 


Parental Knowledge or Attitudes 


.20 b 


.17- .23 


134 


14,745 


Dyadic Interaction 


,30 a 


.27 - .33 


126 


12,179 


Caretaking 


,19 b 


.16- .22 


69 


8,790 


Abuse and Neglect 


.13* 


.11 -.15 


61 


21,689 


All Domains Combined 


.28 


.26 - .30 


282 


36,034 



Note. Because of rounding, there may be slight discrepancies in Ns, both within and 

between tables. . 

a ' b Means with different superscript letters are significantly different from each other. 

‘Mean effect size is significantly smaller than are all other mean effect sizes. 
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Figure 1. Mean effect size and its confidence interval for each outcome 
domain. 
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Mean Effect Sizes and Confidence Intervals: 
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Figure 2. Mean effect size and its confidence interval for each outcome 
domain for low-SES parents compared with all other parents. 
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Fiaure 3. Mean effect size and its confidence interval for each outcome 
domain for adolescent parents compared with all other parents. 
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* Mean effect size for this outcome domain for “at-risk” parents is significantly smaller than is the effect size for all other parents for 
this domain. 
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Figure 4. Mean effect size and its confidence interval for each outcome 
domain for parents meeting abuse and neglect risk criteria compared 
with all other parents. 
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Note. There was an insufficient number of studies in this cell for calculating an effect size. . 

* Mean effect size for this outcome domain for parents with a low birthweight baby is significantly larger than is the effect size for all 

other parents for this domain. 
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Fiaure 5. Mean effect size and its confidence interval for each outcome 
domain for parents with a low birthweight baby compared with all other 
parents. 
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